Humanoid robotics system and methods

ABSTRACT

Systems and methods related to construction, configuration, and utilization of humanoid robotic systems and aspects thereof are described. A system may include a mobile base, a spine structure, a body structure, and at least one robotic arm, each of which is movably configured to have significant human-scale capabilities in prescribed environments. The one or more robotic arms may be rotatably coupled to the body structure, which may be mechanically associated with the mobile base, which is preferably configured for holonomic or semi-holonomic motion through human scale travel pathways that are ADA compliant. Aspects of the one or more arms may be counterbalanced with one or more spring-based counterbalancing mechanisms which facilitate backdriveability and payload features.

RELATED APPLICATION DATA

The present application is a continuation of U.S. application Ser. No.14/814,025, filed Jul. 30, 2015, which is a continuation of U.S.application Ser. No. 14/285,527, filed on May 22, 2014, which is acontinuation of U.S. application Ser. No. 13/084,380, filed on Apr. 11,2011, which claims the benefit under 35 U.S.C. §119 to U.S. ProvisionalApplication Ser. No. 61/322,556, filed Apr. 9, 2010. The foregoingapplications are hereby incorporated by reference into the presentapplication in their entirety.

FIELD OF THE INVENTION

The invention relates to electromechanical and robotic systems foraccomplishing human-scale tasks, and specifically to humanoid andautonomous robotic systems for use in the human environment.

BACKGROUND

Scientists and researchers in countries such as Japan, Germany, and theUnited States have a long history of utilizing electromechanical devicesto assist with tasks such as manufacturing. Most of these systemscomprise a variation of an armlike work interface, and some of them mayhave a base structure which is mobile relative to the coordinate systemof the floor or earth. More recently, a classification of robots oftenreferred to as “humanoid robots” has evolved, with many designers tryingto bring robotics and autonomy deeper into the human living environment.Humanoid robots derive their classification name from human-likequalities that they may imitate or possess. For example, a series ofhumanoid robots developed under the tradename “Asimo®” by HondaCorporation of Japan has demonstrated that small human-like robots cansafely walk, or ambulate, around on two feet with two legs, interactwith humans in the work or home environment, and handle some very lightduty tasks, such as holding a lightweight tray.

Another ambulatory robot from Japan was built and distributed in limitednumbers by Kawada Heavy Industries, Inc., under the name “HRP2”. Indeed,much attention has been focused on solving the humanlike ambulatoryproblem, presumably because human environments are designed forambulatory beings. Other humanoid robotics design efforts have focusedless on ambulation and have developed robots with wheeled bases tofacilitate movement of other important robotic features, such aselectromechanically operated arms or variations of arms. For example,the “STAIR 1” and “STAIR 2” robots developed at Stanford University arerepresentative of a relatively large class of robots wherein a researchteam builds or purchases, for example, from Segway Corporation, arelatively straightforward mobile base, and then mounts a robotic arm onthe top of the base to build a robotics research platform.

A project from Deutches Zentrum fur Luft and Raumfahrt (“DLR”, or the“German Aerospace Center”) called the “Robutler” comprises a movabletable style base with a robotic arm mounted to its top. This system wasconfigured to have laser scanner localization, stereo video cameras, andother sensors to attempt to accomplish what was referred to by DLR as“scene analysis”. A subsequent DLR robotics configuration, termed“Justin”, was significantly more humanoid in its construction andappearance. The Justin humanoid robot has lightweight humanoid arms, ahumanoid sensor head, and a pivotable torso structure. This robot isforce controlled using a series elastic configuration and has expensiveand powerful arms with distally mounted hands comprising fingerlikeelements. Like many of the publicly displayed humanoid robots, it is notfully self contained, in that it has off-board computing and powersystems assisting with its function. Indeed, many humanoid robots aredisplayed or operated with power and/or communications lines trailingbehind due to the complexities of full integration. The University ofKarlsruhe, Germany, has developed a series of humanoid robots called“ARMAR”, which has been displayed with several variations of anonambulatory mobile base, an integrated sensor head, a torso structurewhich may be pivoted to interact with aspects of the environment aroundit, and up to two robotic arms.

It would be desirable for robotics researchers, corporations, and peoplein general to have an integrated humanoid robotics platform fordevelopment and human environment operation that is robust, safe,predictable, fully-integrated, able to mechanically navigate typicalhuman environments, highly programmable, and capable of certainhuman-scale tasks, such as lifting and manipulating objects that arecommonly lifted and manipulated by humans. Such a system is describedherein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1C illustrate orthogonal views of a fully assembled personalrobotics system.

FIGS. 2A-2E illustrate orthogonal schematic views of various aspects ofa personal robotics system at various levels of assembly.

FIGS. 3A-3N illustrate various orthogonal schematic views of variousaspects of a mobile base and body structure assembly for a personalrobotics system at various levels of assembly.

FIGS. 4A-4I illustrate various orthogonal schematic views of variousaspects of an electromechanical caster assembly for a personal roboticssystem at various levels of assembly.

FIGS. 5A-5K illustrate various orthogonal schematic views of variousaspects of an electromechanically mobile head assembly for a personalrobotics system at various levels of assembly.

FIGS. 6A-6M illustrate various orthogonal schematic views of variousaspects of an electromechanical arm pan and counterbalance “turret”assembly for a personal robotics system at various levels of assembly.

FIGS. 7A-7Z illustrate various orthogonal schematic views of variousaspects of an electromechanical shoulder assembly for a personalrobotics system at various levels of assembly.

FIGS. 8A-8R illustrate various orthogonal schematic views of variousaspects of an electromechanical upper arm assembly for a personalrobotics system at various levels of assembly.

FIGS. 8A-8R illustrate various orthogonal schematic views of variousaspects of an electromechanical upper arm assembly for a personalrobotics system at various levels of assembly.

FIGS. 9A-9G illustrate various orthogonal schematic views of variousaspects of an electromechanical forearm assembly for a personal roboticssystem at various levels of assembly.

FIGS. 10A-10D illustrate various orthogonal schematic views of variousaspects of an electromechanical gripper assembly for a personal roboticssystem at various levels of assembly.

FIGS. 11A-11B illustrate schematics pertinent to one embodiment of acontrol infrastructure for a personal robotics system.

FIG. 11C illustrates a schematic of one embodiment of a velocity versusposition controls paradigm.

FIG. 11D illustrates a schematic of one embodiment of a torque versusvelocity controls paradigm.

FIG. 11E illustrates a schematic of one embodiment of a voltage versustime plot for illustrating one embodiment of a power control paradigm.

DETAILED DESCRIPTION

Referring to FIGS. 1A-1C, orthogonal views are shown to illustrate afully assembled personal robotics system (2). As shown in FIG. 1A, amobile base assembly (4) is fixedly coupled to a substantially verticalmain body assembly (8), which is rotatably coupled to each of twoproximal arm turrets (6) which are configured to assist with robotic armcounterbalancing, and also to provide pan rotation for each of the arms.Rotatably coupled to each of the arm turrets is a shoulder assembly (10)configured to facilitate counterbalancing of the arm, and also pitchrotation of the upper arm assembly (14) relative to the proximal armturret (6). The upper arm assembly (14) proximal portion is rotatablycoupled to the shoulder assembly (10), while the distal portion of theupper arm assembly (14) is rotatably coupled to a forearm assembly (16)in a manner that provides a pitch rotation akin to flexion of the humanelbow. The proximal portion of the forearm assembly (16) is rotatablycoupled to the distal portion of the upper arm assembly (14), while thedistal portion of the forearm assembly (16) comprises a differentialdrive mechanism configured to rotate a distally coupled gripper assembly(18) in roll rotation relative to the forearm assembly (16), and/orpitch rotation relative to the forearm assembly (16). A multipart headassembly (12) is rotatably coupled to the superior portion of the bodyassembly (8) and is configured to allow pan rotation of the upperportions of the head assembly relative to the body assembly (8), and/orpitch (or “tilt”) rotation of an uppermost portion of the head assemblyrelative to the body assembly (8). A pitch (or “tilt”) platform at thebase of the head assembly (12) facilitates tilting of a sensor which maybe coupled to such platform, such as a laser scanner.

FIG. 1B depicts another view of the same assembly shown in FIG. 1A, butwith different positioning of several of the components relative to eachother, such as a more prominent tilt rotation of the uppermost portionof the head assembly (12) relative to the body assembly (8). The backside of the body assembly (8) is also shown behind the arm turretassemblies (6). FIG. 1C shows a partial front orthogonal view of thesame assembly shown in FIGS. 1A and 1 B with the upper arm (14), forearm(16), and gripper (18) assemblies repositioned to somewhat highervertical positions of their ranges of motion.

What follows in FIGS. 2A-10D are partial orthogonal schematic views ofvarious aspects of a robotic system such as that depicted in FIG. 1A(2), in various states of assembly and/or dis-assembly. For illustrativepurposes, three dimensional wireframe or solid drawings are utilized toreplicate or illustrate the various aspects of a system such as thatdepicted in FIG. 1A (2), to assist the reader in understanding howvarious parts or portions are configured to work together in a complexelectromechanical configuration.

Referring to FIGS. 2A-2E, various aspects of a mobile base assembly (4)and a body assembly (8) are depicted. FIG. 2A shows a mobile baseassembly (4) rotatably coupled to four electromechanical casterassemblies (three of four shown in FIG. 2A—element 20) configured to bethe system's sole contact with the terrain upon which it moves. Fixedlycoupled to the substantially rectangular prismic shaped movable baseassembly (4) is a body assembly (8). Partially assembled views of a headassembly (12) and two arm turret assemblies (6) are shown coupled to thebody assembly (8), as in the embodiment of FIG. 1A. Shoulder (10), upperarm (14), and forearm (16) assemblies couple the gripper assemblies(18), shown here in schematic form, to the arm turret assemblies (6).

Referring to FIG. 2B, for illustrative purposes, one of the arm andturret assemblies has been removed, revealing additional portions of thebody assembly (8). A similar assembly is shown in a side orthogonal viewin FIG. 2C. Referring to FIG. 2D, all of the arm and arm turretassemblies have been removed, revealing the main structures of the bodyassembly and movable base assembly (4), and the position of theelectromechanical caster assemblies (20) relative to the body (8) andmovable base (4) assemblies. FIG. 2E illustrates the relative placementof the four electromechanical caster assemblies (20) in an assembly suchas that depicted in FIG. 2E, but with the movable base (4) and body (8)assemblies removed.

FIGS. 3A-3N depict further views of aspects of one embodiment of amovable base assembly (4) and body assembly (8). Referring to FIG. 3A,an assembly is depicted wherein the head has been decoupled from theupper portion of the body assembly (8), and portions of the exteriorhousing of such body assembly (8) have been removed. The depictedrobotic system has power and processing systems substantiallycentralized in the mobile base, and communications throughout everyother component to facilitate fine levels of control. One aspect of thecommunications infrastructure, an ethercat bus (36) is depicted at theupper portion of the body assembly (8). Also depicted is a WiFi router(34), an electronic stop (“estop”) switch (24), and a body elevationbellows structure (26) configured to provide a flexible housing to upperportions of the body which may be configured to elevate verticallyrelative to the remaining lower portions. As shown in FIG. 3A, the rearportion of the mobile base (4) comprises an Ethernet interface (30), acooling fan assembly (32), and a power switch panel (28). Fixedlycoupled to the front of the mobile base assembly (4) is a laser scanneror range finder, such as those available from Hokuyo Automatic Co, Ltdof Osaka, Japan, which is housed by the mobile base assembly (4) to havea very broad field of view in front of the mobile base assembly (4).

Referring to FIG. 3B, a side view of a similar assembly is depicted,showing an arm turret support body (38), upper arm turret support member(40), and a lower arm turret support member (42), each of which isconfigured at least in part to support an arm turret assembly (6), suchas those depicted in FIG. 1A, and therefore the remaining arm componentswhich are supported by the arm turret assembly in such a configuration.FIG. 3C illustrates a view of the rear of the mobile base (4) and body(8) assemblies, and shows that each caster assembly (20) in the depictedembodiment comprises a set of two wheels configured to interface withthe pertinent ground terrain.

Referring to FIG. 3D, an orthogonal view of mobile base (4) and body (8)assemblies is shown, in a further deconstructed state relative to FIG.3C. With the external housing skins removed, a main mobile base frameassembly (48) is shown providing a unibody-like lower skeleton to whichmany components are coupled, including two banks of batteries (44)—oneon each side of the mobile base (4). The front of the mobile baseassembly (4) features a large cooling fan air movement passageway/grille(50) that allows heated air to escape after it has been driven inthrough the fan assembly (32) in the back of the mobile base assembly(4), past the middle portion of the mobile base assembly where one ormore computing CPUs are contained, and toward the front cooling fan airmovement passageway/grille (50). Without the exterior housing skins inplace, various aspects of the body assembly (8) are visible, includingseveral aspects of the body vertical support assembly (46), such as anessentially immovable (relative to the mobile base assembly 4) verticalspine assembly (54) which is fixedly coupled to the mobile base frame(48), and a vertically movable spine assembly (52) which is configuredto electromechanically elevated up and down relative to the rest of thebody vertical support assembly (46). Such elevation range of motionallows everything coupled to the vertically movable spine assembly (52)to move vertically with it—including the arm assemblies. Referring toFIG. 3E, to accommodate such vertical motion of one portion of the bodyvertical support assembly (46) relative to another, wiring is passedthrough a flexible wire housing, such as the articulated conduit (56)shown in FIG. 3E to couple wires from the vertical spine assembly (54)to the vertically movable spine assembly (52) and allow for a serviceloop for each wire or conductor that stays neatly in place. Suitablearticulated conduits (56) are available under the tradename “EnergyChain®” from Igus, Inc. of East Providence, R.I., and known as “eChain”.A close up view is shown in FIG. 3F to illustrate the positions of theEthernet port (30), cooling fan assembly (32), and power switch panel(28) at the rear portion of the mobile base assembly (4). Another, moresuperior, close up view is shown in FIG. 3G to illustrate thepositioning of a power distribution board (58) as operably coupled tothe estop button (24), and mechanically coupled to the upper portion ofthe body assembly (8) adjacent the head. FIG. 3H depicts a furtherdisassembled mobile base, essentially comprising the mobile base frame(48), which may comprise sheet metal material subcomponents heldtogether with fasteners to form a unibody-like frame capable ofsupporting relatively large loads. FIGS. 3I through 3N illustratevarious aspects of the electromechanical system configured tocontrollably elevate the vertically movable spine assembly (52) relativeto the vertical spine assembly (54). Referring to FIG. 3I, a bodyelevation drive shaft (62) is shown rotatably coupled, via a lead screwmechanism to the vertically movable spine assembly (52). When a motor(not shown) is activated by a motor controller board (60) mounted to thevertical spine assembly (54), the drive shaft (62) is rotated, causingthe lead screw mechanism to elevate (up or down) relative to thevertical spine assembly (54) and mobile base frame (48). FIG. 3J depictsportions of the body assembly (8) with the body elevation motor,gearbox, and encoder assembly (64) partially visible. FIG. 3K shows aclose up view of the body elevation motor, gearbox, and encoder assembly(64) relative to the drive shaft (62) and body elevation drive frameassembly (68) that houses the body elevation motor, gearbox, and encoderassembly (64) relative to the drive shaft (62). Referring to FIG. 3L,the body elevation motor, gearbox, and encoder assembly (64) isoperatively coupled to a motor pulley (70), which is configured to drivea glass fiber reinforced, flexible timing belt, such as those availablefrom Gates Corporation of Denver, Colo., through a pair of pinch idlers(72) and around a larger driven pulley (not shown in Figure L; element76 in FIG. 3M) which is coupled to the body elevation drive shaft (62).FIG. 3M shows such structures from a partial bottom assembly view. FIG.3N shows the motor pulley (70), belt (74), idlers (72), and drivenpulley (76) in a close up side view. A bearing assembly (78) supportsthe drive shaft (62) with minimal rotational friction.

Referring to FIGS. 4A-41, various aspects of an electromechanical drivecaster assembly are illustrated. FIG. 4A shows one embodiment of a fullyassembled caster assembly, comprising two wheels (82) coupled to acaster turret frame (80). The caster turret frame itself is rotatablycoupled to a caster mounting plate (114) which is configured to bebolted to the main mobile base frame (not shown—element 48 in FIG. 3F,for example). A turret roll orientation motor, gearbox, and encoderassembly (84) is fixedly coupled to the caster turret frame (80) anddriveably attached to dampened drive assembly (86) comprising one ormore elastomer interface pieces (88) to dampen vibrations in thedrivetrain and decrease impulse load peaks. Actuation of the motorassembly (86) causes rotation of the drive assembly (86) and rotation ofa motor pulley (90) coupled to the drive assembly (86). This motorpulley (90) is coupled to a glass fiber reinforced timing belt (92),such as those available from Gates Corporation of Denver Colo., which isrouted around a pulley shaped portion of the caster mounting plate(114). Rotation of the motor causes rotation of the caster mountingplate (114) relative to the caster turret frame (80) and thus roll ofthe entire caster assembly (20) with the exception of the castermounting plate (114) and associated main mobile base frame. FIG. 4Bdepicts a similar assembly from a different perspective. FIG. 4C depictsanother different perspective, which illustrates that a central frameassembly (100) houses three motor controller boards (98)—one for theturret roll orientation motor, gearbox, and encoder assembly (84), andone for each of the individual wheel drive motors. FIG. 4D is anorthogonal view oriented to show a first wheel drive motor, gearbox, andencoder assembly (94) and a second wheel drive motor, gearbox, andencoder assembly (96), each of which is operatively coupled to a drivenpulley (106) comprising one of the wheels (82) using a glass fiberreinforced timing belt (104), such as those available from GatesCorporation of Denver, Colo. Each of the drive motor assemblies (94, 96)is mounted horizontally in the depicted embodiment relative to thewheels (82), with driven pulleys (not shown) coupled to drive axles (notshown) that are mechanically reinforced by bearing assemblies (102)coupled to the caster turret frame (80). FIG. 4E depicts certainelements of the caster turret frame (80) roll (i.e., versus the castermounting plate (114) configuration, along with a schematic diagram ofwires crossing this rotational joint. The caster assembly is configuredto facilitate infinite roll in either direction (clockwise orcounterclockwise) at this roll interface, and this is facilitated byusing slip rings to pass control and power signals across the rotatableinterface between the caster turret frame (80) and caster mounting plate(114). Such slip ring interfaces (116) are used in many rotatableelectromechanical configurations, such as automotive steering wheels.FIG. 4E also illustrates that the motors preferably are mounted to otherstructures using motor mounting assemblies (110) featuring elastomericdamping elements (118) configured to reduce vibration and decreaseimpulse load peaks. FIG. 4F illustrates a cross sectional view through acaster assembly such as that depicted in FIG. 4A, showing the centralpositioning of the wheel axle (112) and motor controller board assembly(98). FIG. 4G shows a partial orthogonal view of the central frameassembly (100) as related to the wheel axle (112). FIG. 4H shows anorthogonal view of a wheel (82) and driven pulley (106) assembly whichmay be held together by four small bolts (not shown). FIG. 4I shows aclose up orthogonal view of a wheel drive motor, gearbox, and encoderassembly (94) coupled to a driven motor pulley (not shown), which iscoupled to a driven wheel pulley (not shown) using a timing belt (notshown). An elastomer-damped driving interface assembly (118) reducesvibrations and peak impulse loads as rotational drive energy istransferred to the motor pulley and wheel; this also decreases theamount of vibrational energy transferred to the mounting assembly (122)and other nearby assemblies generally.

Referring to FIGS. 5A-5K, aspects of one embodiment of a head assembly(12) are depicted. Referring to FIG. 5A, the depicted head assembly (12)comprises a lower assembly (128) that is fixedly coupled to the top ofthe body assembly (element 8 in FIG. 1A), a head scanner assembly (130)that is rotatably coupled to the lower assembly (128) and fixedlycoupled to a scanner sensor (138), such as a laser range finderavailable from Hokuyo Corporation of Osaka, Japan, a middle assemblythat is rotatably coupled to the lower assembly (128) to allow for panrotation of the middle assembly (126) (and upper assembly 124) relativeto the lower assembly, and an upper assembly that is rotatably coupledto the middle assembly (126) to allow for pitch rotation of the upperhead assembly (124) relative to the middle assembly (126). FIG. 5B showsa different orthogonal view of the assembly of FIG. 5A.

Referring to FIG. 5C, portions of the housing have been removed from thelower assembly (128) to reveal further views of the scanner sensor(138), the sensor rotatable member (136) to which the scanner sensor(138) is fixedly coupled, and the sensor mounting base to which thesensor rotatable member (136) is rotatably coupled. Also depicted is amiddle assembly (126) pan actuation drive belt (140). With additionallower assembly housing removed, FIG. 5D illustrates a sensor rotationmotor, gearbox, and encoder assembly (142) which is drivably coupled toa pulley-shaped portion of the sensor rotatable member (136) using afiber reinforced timing belt (144) such as those available from GatesCorporation of Denver, Colo. An eChain articulatable conduit (147) isconfigured to carry wiring to power and control the sensor rotationmotor, gearbox, and encoder assembly (142). Also shown is an eChainarticulatable conduit (146) configured to carry wiring to power andcontrol the motors and sensors above the lower assembly of the head,which are free to pan rotate relative to the lower assembly. Withfurther lower assembly housing removed, a motor controller board for themotor of the sensor rotation motor, gearbox, and encoder assembly (142)is depicted coupled nearby.

With additional lower assembly housing removed, FIG. 5F illustrates thehead pan rotation motor, gearbox, and encoder assembly (152) which iscoupled to a motor pulley (156) that drives a fiber reinforced timingbelt (140), such as those available from Gates Corporation of Denver,Colo., through a set of idlers (158) and around a relatively largedriven pulley (160) which causes the middle and upper assemblies of thehead assembly to pan rotate relative to the lower assembly and bodyassembly.

With additional portions of the lower assembly removed, FIG. 5G depictsa front view of the upper head assembly, which, in this embodiment,features a set of stereo cameras (162), one or more projectors (164),and one or more other imaging cameras or other sensors (166). Theprojectors (164) may be structured light projectors or unstructuredlight projectors—either type may be used to project some texture againstitems in the field of view of the stereo cameras (162) to facilitateenhanced object and three dimensional image capture analysis. Multiplepairs of stereo cameras (162) may be utilized, which may combine widefields of view, narrow fields of view, relatively high resolution,relatively low resolution, structured or unstructured light analysis,etc. Cameras may reside not only coupled to aspects of the head assembly(12), but also coupled to other assemblies, such as forearm or gripperassemblies (16, 18). For example, in one embodiment, a gripper assemblymay comprise a simple camera or laser reflection configuration to assistwith grasping small elements such as wires in the fingers of suchgripper, which conventionally is a challenging task.

FIG. 5H shows a further disassembled head assembly wherein the upperhead assembly (124) pitch actuation motor, gearbox, encoder assembly(154) is visible, along with an eChain (148) to safely channel power,control, and sensor wiring from the pitch rotating portions of the upperhead assembly (124) to other portions of the robotic system. FIG. 5Ishows a further disassembled head assembly wherein a motor pulley (168)coupled to the motor of the head assembly (124) pitch actuation motor,gearbox, encoder assembly (154) is coupled to a relatively large drivenpulley (174) with a fiber reinforced timing belt (172), such as thoseavailable from Gates Corporation of Denver, Colo., in a configurationwherein the motor of the head assembly (124) pitch actuation motor,gearbox, encoder assembly (154) may electromechanically pitch rotate theupper head assembly (124) about a pitch rotation axle (176), as shown inFIG. 5J and 5K.

FIGS. 6A-6M depict aspects of the proximal arm pan and counterbalancingturret assembly (6). Referring to FIG. 6A, a turret mounting assembly(7) fixedly coupled to the body assembly (element 8 of FIG. 1A)facilitates rotation of the turret assembly (6) relative to the bodyassembly. The same assembly of FIG. 6A is shown from a differentorthogonal view in FIG. 6B to illustrate the “U” type shape of theturret mounting assembly (7), and the placement of the turret pan motorcontrol board (178) as coupled to the turret mounting assembly (7). FIG.6C shows a turret mounting assembly (7) without a turret assembly, toillustrate the position of the turret pan motor, gearset, and encoderassembly (180) as fixedly coupled to the turret mounting assembly (7).Referring to FIG. 6D, a partially disassembled turret assembly (8) isdepicted, illustrating the three main outer structures—the top capmember (182), bottom cap member (184), and middle turret structuralassembly (186) including two shoulder interface tabs geometricallyconfigured to support a shoulder assembly, as described below. Amounting foot (214) and an upper mounting member are configured to befixedly coupled to the turret mounting assembly (7), and are rotatablycoupled to the bottom cap member (184) and top cap member (182),respectively. The interface (202) between the top cap member (182) andupper mounting member (216) comprises a rotational range of motionlimiting groove into which a toe of the upper mounting member fits. Thegroove terminates with elastomeric cushions (204) to limit suchrotational range of motion (i.e., the foot of the upper mounting memberhits the cushions and this interference prevents further pan rotation ofthe turret assembly (6). The shoulder pitch motor, gearbox, and encoderassembly (194) is depicted coupled to the upper aspect of the middleturret assembly (186) on the turret main frame structural member (242).For spatial efficiency reasons, the motor controller board (192) forthis shoulder pitch motor, gearbox, and encoder assembly (194) iscoupled to the inferior aspect of the middle turret assembly (186) onthe turret main frame structural member (242). As described below, first(188) and second (190) spring cans are coupled to the turret main framestructural member (242). Springs within these spring cans (188, 190) arecoupled to an output belt which is coupled to an elevator assembly(200). The elevator assembly is free to elevate up and down vertically,and to rotate about a vertical elevator rod (248 in FIG. 6L, forexample); it is also coupled to a gimbal assembly which terminates in anupper arm counterbalancing interface rod (196), through which the springcans (188, 190) may ultimately serve to counterbalance gravitationalloads on the upper arm and forearm assemblies (14, 16). FIG. 6E depictsa rear view of a partial arm turret assembly, including the uppermounting member (216), top cap member (182), middle turret assembly(186), bottom cap member (184), and mounting foot (214) member. FIG. 6Fillustrates a close up orthogonal view of a top cap member (182), uppermounting member (216), and the groove/toe rotation-limiting interface(202) with elastomeric cushions (204) to decrease vibrations and peakimpulse loads. FIG. 6G depicts an underside view of the assembly in FIG.6F, with the center of turret pan rotation (212) identified, as well aselastomeric cushion cap members (206) and a pan drive belt terminationinterface (208) and pan drive belt termination/adjustment interface(210). FIG. 6H illustrates a close up orthogonal bottom view of a bottomcap member (184) rotatably coupled to a mounting foot (214). FIG. 6Iillustrates in cross section a heavy duty bearing interface (218)between the bottom cap member (184) and mounting foot (214). A similarlyheavy duty bearing interface is interposed between the rotatably coupledtop cap member (182) and upper mounting member (216).

FIGS. 6J-6M illustrate aspects of one embodiment of a spring-balanced(i.e., actuated using a mechanical spring assembly as opposed to anelectromechanical actuator—so that the gravity compensation isoperational regardless of the power state of the associated roboticsystem) gravity compensation mechanism. Referring to the cross sectionalview of FIG. 6J, the first spring can (188) contains a compressedcompression spring which may be adjusted using a spring adjustment bolt(222). Similarly, the second spring can (190) contains a compressedcompression spring which may be adjusted using a spring adjustment bolt(220). Compressive loads from the springs are applied against thedepicted pistons (219, 220) to cause tensile loads in the spring canoutput belts (228, 230). These output belts (228, 230) wrap around andare terminated upon belt/pulley interface members (232,234). The firstbelt/pulley interface member (232) is fixedly coupled to a pulley (235)which operates the final output belt (240), which is coupled to theelevator assembly (200). Referring to the less-deep cross-sectional viewof FIG. 6K, the second belt/pulley interface member (234) is coupled toa pulley (236) which is coupled by a timing belt (238), such as thefiber-reinforced timing belts available from Gates Corporation of DenverColo., to an additional pulley (238) which is coupled to the pulley(235) described above which is coupled to the first spring can (188) andfinal output belt (240). Thus, both spring cans (188, 190) work togetherthrough the depicted mechanism to apply a layered counterbalance loadingschema (i.e., “layered” in that two different load versus deflectioncurves are being overlaid—one from the first spring can, one from thesecond spring can) to the elevator assembly (200) through the finaloutput belt (240), and this layered counterbalance loading schema isdirected to a counterbalancing member (310) residing within the upperarm assembly, as described below, through the elevator assembly (200),gimbal assembly (198), and interface rod (196). The gimbal assembly(198) provides freedom of motion as the elevator and counterbalancingrod are moved relative to each other, while remaining coupled throughthe gimbal assembly (198). The spring cans (188, 190) are designed toprovide gravity counterbalancing for the upper arm and forearmassemblies—to assist in making these assemblies “float” or function asthough they have less gravity, even when the power to the robotic systemhas been turned off. As opposed to the spring based counterbalancingmechanism disclosed in U.S. patent application Ser. No. 12/626,187 (U.S.Publication No. 2010-0243344), incorporated hereby by reference in itsentirety, the spring cans in the subject embodiment are not activelyadjusted using takeup drums during counterbalancing. A similar four barlinkage arrangement is utilized as in the aforementioned incorporatedapplication, but in the subject embodiment, the spring cans are adjustedto counterbalance the forearm assembly (16) and make it substantially“float” relative to gravity. Then an adjustment between the upper armcounterbalance interface rod (196) and the counterbalancing member (310)to lengthen or shorten the dimension of this interface is made to getthe upper arm assembly (14) to substantially “float” relative togravity. With such a configuration, both the upper arm (14) and forearm(16) assemblies may be gravity compensated. In one embodiment, thenominal load in each of the spring cans is about 130 lbs, and this maybe adjusted up to about 40% by modifying the relationship between thethe upper arm counterbalance interface rod (196) and thecounterbalancing member (310). The cam geometry of one of the pulleys(236) is configured to make the system of two spring cans operate likeone constant force acting to counterbalance the arm assemblies. FIGS. 6Land 6M illustrate non-cross sectional views of similar assemblies toshow the pertinent mechanism portions three-dimensionally relative toeach other.

Referring to FIGS. 7A-7Z, aspects of one embodiment of a shoulderassembly interposed between an arm turret assembly and an upper armassembly are depicted. As shown in FIG. 7A, an upper mounting member(216), top cap member (182), and shoulder interface tab (244) of an armturret assembly are depicted. Also depicted is an upper arm assembly(14). In between the arm turret and upper arm components is a shoulderassembly comprising a mechanism for facilitating pitch rotation of theupper arm assembly relative to the arm turret assembly, and also rollrotation of the upper arm assembly relative to the arm turret assembly.FIG. 7A shows an upper arm roll motor, gearbox, and encoder assembly(250), as well as a motor control board (252) for this motor that iscoupled to a shoulder frame (254). FIG. 7B shows a slightly closerorthogonal view depicting not only the upper arm roll motor, gearbox,and encoder assembly (250), but also an associated motor pulley (256),pair of pinch idlers (258), and upper arm roll actuation belt (260),such as the fiber reinforced timing belts available from GatesCorporation of Denver Colo., routed around the motor pulley (256) andidlers (258) as shown, and around a driven pulley member configured tointerface with a proximal portion of the associated upper arm assembly(14) to result in electromechanically operated roll rotation of theupper arm assembly relative to the shoulder assembly. Referring to FIGS.7C-7E, such upper arm rotation is shown incrementally over time as theupper arm is rotated about its roll axis (264) through this series ofthree drawings. Further, this set of figures also demonstrates the upperarm pitch axis (262).

FIG. 7F depicts a shoulder assembly (266) coupled to an arm turretassembly (6) without an upper arm assembly coupled to the shoulderassembly (266). FIG. 7G illustrates a closer orthogonal view with someof the outer members made transparent. The aforementioned shoulder pitchmotor, gearbox, and encoder assembly (194) drives the depicted motorpulley (280), which drives a belt (274), such as one of aforementionedfiber reinforced timing belts, through a pair of idlers (268) and arounda shoulder pitch driven pulley (270). A belt tensioner assembly (276) isdepicted coupled the pulley (270). Operation of the motor drives themotor pulley (280) to drive the belt (274) to cause rotation of thepulley (270) and therefore pitch rotation of the shoulder assembly (266)about the pitch rotation axis (262). FIG. 7H depicts a furtherdisassembled assembly, including a coupling member (278) configured tobe fixedly coupled to one of the shoulder interface tabs (244) of thearm turret assembly (6), and rotatably coupled to the remainder of theshoulder assembly. Pitch range of motion is restrained with a portion ofthe coupling member (278) which rides through a groove formed in thedriven pulley (270) and ends at each extreme of the groove with anelastomeric member configured to dampen vibrations and decrease maximumimpulse loads. FIG. 7I illustrates these elastomeric dampeners (290), aswell as a homing sensor configuration comprising a light beam sensor(286) and a ridge (288) formed into the pulley (270) with adiscontinuity or discontinuation at a point predetermined to be a “home”position for this shoulder pitch range of motion. In other words, asdescribed below, the control system may be configured to sense a dark(with the intermediate ridge blocking transmitted light) to light(without the intermediate ridge blocking—so light is transmitted to theother side of the sensor) transition and position such joint at aspecific range of motion relative to such transition as a “home”position for such joint. Indeed, preferably each joint of the subjectrobotic system is equipped with such a home position sensingconfiguration. In one embodiment, when a home position transition up ordown is sensed at the pertinent motor controller board, an encoder valueis latched at that exact time—so even though the system may only have amillisecond of resolution, it is known, even if 20 encoder ticks aremoved during that millisecond, precisely at which encoder tick the homeposition flag was hit—a valuable precision benefit.

FIG. 7I also illustrates a shoulder frame (282) which couples shoulderpitch activity to upper arm roll activity, as driven by the upper armroll motor, gearbox, and encoder assembly (250), motor pulley (256),drive belt (260), pair of idler pulleys (258), and driven upper arminterface member pulley (284). FIG. 7J depicts a different orthogonalview of the sensor (286), ridge (288), coupling member (278), and eChain(292) wiring conduit to facilitate communications and power transferbetween the arm turret assembly (6) and the shoulder assembly (266).FIG. 7K is a further close up orthogonal view of portions of theassembly featured in FIG. 7J. Referring to FIG. 7L, anothershoulder-turret interface member (294) is depicted, which is fixedlycoupled to the interface tabs (244) of the turret assembly (6). FIG. 7Mdepicts the shoulder pitch motor, gearbox, encoder assembly (194), ascoupled to the arm turret assembly (6), operably coupled to causeelectromechanical shoulder pitch rotation as described above. FIG. 7Ndepicts a partial orthogonal three dimensional view to illustrate therelative positioning of the shoulder pitch motor, gearbox, encoderassembly (194) relative to various components of the shoulder assembly.A substantially cylindrical shoulder housing (272) provides a structuralframe for many elements, in addition to the shoulder frame (282). Apartial side orthogonal view is depicted in FIG. 7O, wherein theshoulder pitch driven pulley (270), drive belt (274), belt tensioningassemblies (276), and eChain (292) are shown. A further disassembly isdepicted in FIG. 7P, and in a different orthogonal view in FIG. 7Q,showing an upper arm roll eChain (296) configuration placed to allow thetransfer of signals between the shoulder and the upper arm. A tunnelvolume (298) for a counterbalancing bar, as well as certain wires, isdepicted through the shoulder assembly in FIGS. 7Q and 7R. FIG. 7S showsa further disassembled view clearly depicting the upper arm roll motor,gearbox, encoder assembly (250) and associated motor controller board(252); FIG. 7T depicts a wireframe version of a similar partialassembly. FIG. 7U depicts a partial cross sectional view of a shoulderassembly to illustrate the shoulder pitch rotation axis (262) and heavyduty bearing (300, 302) interfaces between the shoulder turret interfacemember (294) and shoulder frame (282), and between the shoulder frame(282) and coupling member (278). FIGS. 7V-7X further illustrateaforementioned portions of the shoulder assembly pertinent toelectromechanically driven upper arm roll rotation. FIG. 7Y illustratesa partial orthogonal close up view of an upper arm coupling interfacemember (284) having a homing ridge (304) defined thereon, and configuredto rotate through a sensor (306), such as a light transmission sensor—tofacilitate home positioning with a predetermined dark-light sensedtransition, as described above in reference shoulder pitch homing. FIG.7Z illustrates a partial cross sectional view to show that roll of theupper arm coupling interface member (284) relative to the shoulder frame(282) is facilitated by a heavy duty ring bearing (308) interposedbetween such structures.

FIGS. 8A-8R depict aspects of one embodiment of an upper arm assembly(14). A complete upper arm subassembly (14) is depicted in orthogonalviews in FIGS. 8A, 8B, and 8C having exterior skin structures (314)intact as well as a distal interface member (316) for coupling with theproximal end of a forearm assembly (16), and a proximal interfacestructure (312) for interfacing with the distal end of a shoulder (266)assembly. The proximal end of a counterbalance member (310) is shownexiting the proximal end of the upper arm assembly (14), with enoughlength to pass through a shoulder assembly (266) to a coupling with anupper arm counterbalance interface rod (196) extending from the armturret assembly (6). Referring to FIG. 8D, an orthogonal view into theproximal end of the upper arm assembly (14) reveals more of thecounterbalance member (310) as it extends distally. An elbow pitchmotor, gearbox, and encoder subassembly (318) is also visible as it iscoupled to the main frame element (820) of the upper arm assembly, asshown also in FIG. 8E wherein an upper arm assembly (14) is depictedwithout the external skin structures. Referring to FIGS. 8F-8L, withportions of the frame structure (320) and other structures removed, theupper arm counterbalance interworkings are shown. Referring to FIG. 8F,the counterbalance member (310) is rotatably coupled to a proximalpulley (328) at a first coupling point (322). The counterbalance member(310) is then rotatably coupled to an intermediate pulley (331), andfinally/distally it is rotatably coupled to a distal pivot point (324)on a distal timing pulley (330). A timing belt (326), such as thoseavailable in fiber reinforced form from Gates Corporation of Denver,Colo., maintains a geometric/kinematic relationship between thecounterbalance member (310) and each of the two pulleys (328, 330), in amanner similar to that described in the aforementioned incorporated byreference application U.S. patent application Ser. No. 12/626,187 (U.S.Publication No. 2010-0243344). A belt tensioner subassembly (332) isshown on each of the pulleys (328, 330). Downward gravity based loadsassociated with the masses of the upper arm assembly (14) and forearmassembly (16) are counterbalanced with loads imparted to thecounterbalance member (310), which is kinematically operated as a fourbar linkage, such that every portion of the counterbalance member (310)moves in the same circular pattern. Thus the mechanism has a very cleantranslator of load from the distal upper arm to the proximal end of thecounterbalance bar (310), and the system is configured to apply loads tothis proximal end to gravity compensate the forearm assembly (loadsapplied to the proximal end of the counterbalance bary 310 will torquethe distal pulley 330, which urge the forearm assembly up versusgravity); additionally, as discussed above in reference to the armturret assembly (6), another gravity compensation loading schema isoverlaid or “superimposed” onto the counterbalancing for the forearm, tocounterbalance the gravity loads upon the upper arm assembly (14). Theelbow flexion range of motion at the distal pulley (330) is about 130degrees in the depicted embodiment, and at the end of this range ofmotion, there is a singularity wherein but for the timing belt (326),the two pulleys (328, 330) and counterbalance member (310) could becomemechanically jammed relative to each other with one pulley rotating inan opposite direction of the other. The timing belt (326) rotates bothpulleys together through this singularity and prevents the distal pulley(328) from rotating in the wrong direction. If the kinematics were notrotating through this singularity, the timing belt would not be neededto enforce stability. Thus, with the counterbalance member (310) andassociated gimbal/elevator interface provided as part of the proximalarm turret assembly (6), a fully general configuration for balancing twolinks (upper arm 14 and forearm 16) is presented.

Referring to FIGS. 8G, 8H, and 8I, sequential motion of thecounterbalance member (310), and proximal (330) and distal (328) pulleysover time is depicted to depict the associated counterbalancingkinematics as portions of a 4-bar linkage.

FIG. 8J illustrates the opposite side of the mechanism from thatdepicted in FIGS. 8F-8I. Referring to FIG. 8J, the elbow pitch rotationmotor, gearbox, and encoder assembly (318) is driveably interfaced witha set of bevel gears (338) to drive a driven motor pulley (340) andassociated timing belt (344), such as the glass reinforced timing beltsavailable from Gates Corporation of Denver, Colo., though a set of idlerpulleys (342) and around a relatively large distal driven pulley (334)which is coupled to the distal pulley (330) on the opposite side of theupper arm assembly (14). Thus the elbow pitch rotation motor, gearbox,and encoder assembly (318) actuates rotation of both the proximal (328)and distal (330) pulleys of the aforementioned timing andcounterbalancing arrangement. Also shown in FIG. 8J are belt tensioningsubassemblies (346) and elastomeric bumpers (336) to enforce the maximumrotational range of motion of the elbow pitch rotation. Referring toFIG. 8K, the kinematic relationships of the driven pulley (334), distalpulley (330), and proximal pulley (328) are shown, along with the timingbelt (326). A further deconstructed view in FIG. 8L shows additionalaspects of the counterbalance member (310) and its coupling to thevarious other counterbalancing parts. Also shown in FIG. 8L is thedistal housing member (348) and forearm roll mechanism comprising aforearm roll motor, gearbox, and encoder assembly (350), associatedmotor controller board (352), and distal forearm mounting interface(316). The further deconstructed orthogonal view of FIG. 8M illustratesthe distal pulley axle interface member (354) which is configured tocouple the driven distal pulley (334) with the distal counterbalancingpulley (330). FIG. 8N depicts a further deconstructed view wherein theforearm roll motor, gearbox, and encoder assembly (350) is coupled to amotor gear (356), which is directly interfaced with a larger forearmroll rotation driven gear (358), which is coupled directly to the distalforearm mounting interface member (316). Additional orthogonal views ofsuch assembly are depicted in FIGS. 8O-8Q. FIG. 8R illustrates a furtherdeconstructed view to show that the forearm roll driven gear (358) iscoupled to a forearm roll axle (360) which is rotatably coupled throughthe central axle junction member (354) to an additional forearm rollencoder (362). The central axle junction member (354) is coupled to aslip ring wiring interface configured to pass electrical signals betweenthe forearm and upper arm, to allow infinite roll rotation in eitherdirection of the forearm assembly (16) relative to the upper armassembly (14). In one embodiment of the system, there are three infiniterotation slip-ring enabled joints: here between the forearm and upperarm, at the casters, as described above, and between the forearmassembly (16) and gripper assembly (18), as described below.

Referring to FIGS. 9A-9G, aspects of one embodiment of a forearmassembly (16) are depicted. Referring to FIG. 9A, an orthogonal view ofa forearm assembly (16) is depicted, comprising a distal grasperinterface assembly (364) configured to mechanically couple with agrasper assembly (18), as depicted in FIG. 9B, with a grasper interfaceclamp assembly (366) comprising portions of each associated assembly(16, 18) clamped against each other, as illustrated in FIG. 9C.Referring to the cross sectional view FIG. 9C, the interface assembly(366) comprises portions (368, 370) of each assembly clamped togetherwith bolts and flat-faced fittings configured to not allow forrotational slop at the interface (366). A gripper roll axle (372) isshown fitted though the middle of the interface (366). Referring toFIGS. 9D-9G, the distal forearm comprises a differential drive mechanismconfigured to pitch and/or roll the distal differential drive member(386) which may be coupled directly to a grasper assembly (18).Referring to FIG. 9D, a first differential drive motor, gearbox, andencoder assembly (374) is coupled to a bevel gear (378), which isinterfaced with a driven gear and pulley assembly (382) that is coupledinto the distal differential drive member (386) mechanism. Similarly, asecond differential drive motor, gearbox, and encoder assembly (376) iscoupled to a bevel gear (380), which is interfaced with a driven gearand pulley assembly (384) that is coupled into the distal differentialdrive member (386) mechanism. FIG. 9E depicts a cross sectional viewwith heavy duty bearings (388, 390) providing efficient rotationalinterfaces for portions of the differential drive mechanism, includingthe gripper roll axle (372). A partial orthogonal view is shown in FIG.9F to illustrate the positioning of one of the differential drive motor,gearbox, and encoder assemblies (376) relative to the various portionsof the differential drive assembly. FIG. 9G illustrates an additionalpartial orthogonal view to show that a motor controller board (392, 394)is coupled adjacent each of the differential drive motor, gearbox, andencoder assemblies (374, 376). In addition, a camera window (396) isprovided for a small digital camera module to be positioned to have afield of view capturing motion of the gripper interface assembly (364)and associated gripper assembly (18).

Referring to FIGS. 10A-10D, aspects of one embodiment of a gripperassembly (18) are illustrated. Referring to FIG. 10A, a gripper assembly(18) comprises one portion (370) of the gripper/forearm interfaceassembly at its proximal forearm interface (408). Coupled to the twodistal fingers (404, 402) comprising the grasper are finger pressuresensor arrays (398, 400) comprising 22 or more sensors at each fintertipwhich not only monitor fingertip interfacial pressures, but alsopressures around the sides of the fingertips. Such sensor arrays (398,400) are available from Pressure Profile Systems, Inc., of Los Angeles,Calif. Five or more palm interface pads (406) are configured to providepalm-like interfacing with objects grasped by the gripper. Referring toFIG. 10B, a partially disassembled gripper assembly is shown withportions of the outer housing assembly (418) hidden away to illustratethe four bar kinematic linkage comprising the grasping mechanism, andits actuation means. Referring to FIG. 10B, each of two connectingmembers (420, 422) is fixedly coupled to a spur gear (412, 410). Theconnecting members (420, 422) are also rotatably coupled to the housing(418) at pivot points (488, 490), and rotatably coupled to the fingermembers (404, 402) at finger pivot points (492, 494). A grasping motor,gearbox, and encoder assembly (416) and nearby motor controller board(414) actuates grasping activity by urging the connecting members (420,422) toward each other. Referring to FIG. 10C, two sets of connectingmember interface pins (432, 434) provide the mechanical interfacebetween the grasping motor, gearbox, and encoder assembly (416) and theconnecting members (420, 422). Referring to FIG. 9D, one embodiment of agrasping motor, gearbox, and encoder assembly (416) is shown in greaterdetail, in addition to other aspects of the grasper linear drivemechanism. Referring to FIG. 10D, actuation of the motor of the graspingmotor, gearbox, and encoder assembly (416) causes rotation of a motorpulley (424), which drives a timing belt (428), such as the fiberreinforced timing belts available from Gates Corporation of Denver,Colo., to rotate a driven pulley (426), which causes a ball screwassembly to move one set of connecting member interface pins linearlyrelative to the other set (434).

Referring to FIG. 11A, a controls architecture diagram is depicted toillustrate how the various aspects of one embodiment of a roboticsystem, such as that depicted in FIG. 1A, may be controlled. At thecenter of the controls architecture are one or more computers (462, 464)configured to share the processing loads. As discussed in reference tothe mobile base above (4), these devices preferably reside near thecenter of the mobile base unit, adjacent the power supply and coolinginfrastructure. In one embodiment, the computers (462, 464) comprise X86type personal computers running Linux operating systems. In oneembodiment, each has two or more multicore processing chips, such asthose available from Intel Corporation of Santa Clara, Calif. Eachcomputer system also preferably has a relatively large amount of randomaccess memory and hard drive or flash memory drive capacity. In oneembodiment, each computing system also comprises a discrete internaldrive configured to store the operating system, as well as a removable3.5-inch drive configured for rapid movement of large amounts of data onor off board the robotic system. In one embodiment, the first computer(462) is configured to boot Linux from its hard drive, and to triggerthe second computer (464) to boot using a “netboot” procedure. A 16 bitgigabit switch (460) functions as a communications backbone of sorts,with connections to both computers. In one embodiment, either of thecomputers (the second, 464, in the depicted embodiment) may have two ormore connections to the switch (460) for load balancing. Also coupled tothe switch are one or more Ethernet camera devices (446), the powercontrol board (444), and “service port” communications infrastructure,in the form of a wireless access port (500), and an rj45 connector(502); these may also be referred to as the “WAP service port” and the“wired service port”, respectively. As described above, the firstcomputer (462) may be configured to boot from its own 3.5-inch drive andstart serving netboot to the second computer (464). As the secondcomputer boots up, it may be configured to go out on the network andavailable bios to see what is available, see the first computer servingnetboot, and boot from the first computer (462). One advantage of suchas booting configuration is that only the first computer (462) need beupdated (for example, with new software, new users, etc), and this willupdate the second computer (464) and keep it compatible. In other words,the netboot image that the first computer (462) sends over to the secondcomputer (464) causes the second computer (464) to remote mount thefirst computer (462) for the root file system—so the two computers endup running not only the same version of Linux, but also the same Linuxroot file system, with the exception of a few small differences, such asdifferent host configuration and network configuration. A stackableunification software file system such as UnionFS® may be utilized tofacilitate the software unification between the two computers. Once thetwo computers have been started up, they are both running Linux, andconfigured such that operators of the robotic system will not noticethat they are running off of one another. Standard users preferably areconfigured, and an operator may connect to the system directly as auser, or over various web-based interfaces. In one configuration, thecomputers are configured to serve a web interface directly from therobot. In other configurations, web interfaces may be served fromoff-board systems and configured to communicate to the robot using arobotic operating system, such as the ROS operating system availablefrom Willow Garage, Inc. of Menlo Park, Calif., which is a programmingframework for robotics that facilitates the handling of issues such asinterprocess communication and the build system.

In another embodiment, one computing system may stand alone for centralprocessing requirements, or two could be utilized with the secondgenerally in a power-saving standby mode and being configured to onlypower up when demands on the first computer pass a certain threshold(redistribution of some of the processing then to the second computercould be handled by running the pertinent processes as virtual machines,so they would be migratable back and forth between computing systems).With each computing system representing a significant robotic systembattery draw, such a flexible architecture may be quite desirable.

In one embodiment, many (on the order of 100 or more) processes will berunning at once on the one or more computing systems, and theseprocesses will be handling many disparate issues, including the realtimeprocess. In one embodiment, all of the processes may be run with what isknown as an “RT-preempth patch” to the Linux kernel, which essentiallyrepresents Linux running with changes in a configuration wherein thereis nothing that will take an unbounded amount of processing time for thekernel. In other words, in such a configuration, everything ispreemptable and a scheduler may be set up to always run the highestpriority process. With such a configuration, the one or more computingsystems may be set up to run the entire controls schema, as well asother processes in real time. Many processes may be run in “soft realtime”, as modern computing systems are essentially fast enough, and itis preferable to have them in fault tolerant modes. With the computingsystem running the various processes, it can be configured tocommunicate over the aforementioned ROS operating system and run thedesignated set of realtime controllers. For example, in one embodiment,a controller is configured to compute odometry information to track therobot's movement over time, and publish this information out. Anotherpublishing node may be configured for a Hokoyo laser scanner devicewhich may be plugged into one of the computers (say the first computingsystem, 462) where the driver for such device may reside. Such a nodewould may be configured to publish scan information from the laserscanner device. The odometry and scan information may be run intolocalization and mapping processes to make a map available as a latchedtopic, for example; subscribing devices will see this data stream alwaysstarting with the last item that was sent. This is a means for puttingpersistent data out on the network so any other device or process canreceive it—and without having to send the data out all of the time(i.e., a query will result in whatever information has been requested).Thus such a configuration would be publishing, and in the aggregate, thesystem would build up a significant amount of interrelated informationand componentry.

As discussed above, the switch (460) is at the center of thenon-realtime communications network, which generally comprises astraightforward ethernet configuration wherein the WAP service port(500), the wired service port (502), power control board (444), Ethernetcamera devices (446) and computers (462, 464) are operably coupled. Inthe depicted embodiment, the first computer has an additional networksegment which may be referred to as the “network port” infrastructure,wherein a network wireless access port (454), a network rj45 connection496), and a network SIM card interface (498) are operably coupled to thefirst computer (462) through a virtual private network, or “VPN”,interface (506). Whereas the service port infrastructure (500, 502) maybe utilized to simply access other devices on the network (i.e., it isconfigured to be a network node that may be utilized by an outsideresource, such as a laptop computer workstation, at any time fordebugging, to reprogram one or more of the computers, etc.; the robotessentially serves DHCP to the outside resource, so the resource becomeslike part of the robot network), the network port infrastructure isconfigured to be utilized to provide the robotic system with Ethernet.In other words, the network port infrastructure is configured to connectwith an external network, such as a building network in the buildingwhere the robot happens to reside or a mobile wireless network nearbyusing the SIM card interface (498) and appropriate SIM card for thepertinent mobile network (i.e. 3G, LTE, WiMax, etc.), to be able toreceive data and access to off-board resources, such as a base computingstation (504). In one embodiment, so long as the first computer (462) isable to receive some packets through its VPN (506) from the off-boardbase station computing system (504) through that system's VPN (508),regardless of the kind of network transfer, then networking connectivitymay be established.

Referring again to FIG. 11A, also attached to the first computer (462)through one or more USB interfaces are various devices. In the depictedembodiment, for example, a Hokuyo laser scanner (532) and camera device(534) may be connected as USB devices (528) to the USB interface (526)of the first computer (462). Similarly, a joystick (536), such as thoseavailable from Sony Corporation under the tradename PlayStation®, PDA(538), such as those available from Apple Computer, Inc. under thetradenames iPhone® and iPad®, or other Bluetooth-enabled devices may beoperably coupled with the USB interface (526) of the first computer(462) using a Bluetooth adaptor (530) configured with software code thatis always running that knows to connect to certain Bluetooth devices.Also shown in FIG. 11A, one or more displays (466) may be operablycoupled to video outputs of the computing systems (462, 464) tofacilitate operation of the computers, display of graphics relatedthereto, etc.

Referring again to FIG. 11A, separate from the above non-realtimecommunications Ethernet network is a realtime network operation (456)which runs on 100 megabit Ethernet and is operatively coupled to all ofthe motor control boards (438, 440, 442), the robot-head-mounted textureprojector (452), and any other sensors which may be coupled into therealtime network (456). In one embodiment, an Ethercat “hub” (524)architecture is utilized, as described, for example, by BeckhoffCorporation, the organization that developed Ethercat. An Ethercat hubbehaves somewhat differently than a conventional Ethercat passthroughconfiguration, such as those known as “token ring” configurations.

As described above in reference to the robotic system hardware, motorcontrol boards, such as those shown in FIG. 11A functionally coupled into the realtime network (456), are positioned all over the robot—toenable precision control of joints and other devices. Several motorcontrol board types may be utilized in the subject robotic system. Onemain type is configured to connect to various pieces of hardware (436),such as one motor, one encoder, a limit switch (such as the dark tolight transition sensing switches described above in reference tomovable joints and homing positions), and a trigger output (for example,in the forearm assembly 16, a motor control board that controls thejoint movement is also utilized to trigger a camera exposure withmillisecond precision). Such configuration preferably comprises a fieldprogrammable gate array, or “FPGA”, device, an Ethercat communicationchip (which may also be implemented as an FPGA core in anotherembodiment, as opposed to a discrete chip), an H-bridge (an electroniccircuit which enables a voltage to be applied across a load in eitherdirection, generally to allow a DC motor to run forward or backward; inone embodiment the H-bridge may be an integrated circuit; in anotherembodiment it may comprise discrete components), one or more inductors(in one embodiment, to facilitate DC current control, the H-bridge maybe filtered out), a current sensor, input voltage sensor, output voltagesensor, and one or more temperature sensors (for example, in oneembodiment, a motor controller board has two temperature sensors—one forboard ambient, and one for the H-bridge; in another embodiment, athermocouple input is provided on the motor control board to monitor athermocouple lead configured to directly monitor motor temperaturedirectly and allow for motor model adjustment accordingly; without sucha direct motor temperature measurement configuration, motor temperaturemay be backed out of calculations based upon resistance/temperaturemodels). The current control loop preferably is controlled by the FPGA,and an operator with the right access can reset gains, change thecontrol loop, or even the entire control scheme (i.e., voltage mode,current mode, variable control directly on the FPGA, etc.), all byremotely programming the FPGAs. Thus this main type of motor controllerboard configuration is quite capable, and is utilized throughout theabove described robotic system.

The motor controller boards for the gripper assemblies (18) are a bitdifferent, in that they also comprise a three axis accelerometer (so allof that data comes back synchronously in the datastream back to thecomputing systems) and an LED light that is brought from the motorcontroller board out to the surface of the gripper using a light pipe—toallow for certain movement/timing calibration and/or testing maneuvers.For example, there are a few scenarios wherein it is useful to connectbetween the realtime and nonrealtime networks. In one embodiment, amotor controller board (440) may be used to trigger (450) a device suchas an Ethernet-connected camera (446)—to capture an image and also knowexactly when that image was captured. In such a configuration, thecamera device (446) itself still resides on the nonrealtime network, andthe motion control is happening on the realtime network. In a relatedembodiment, an LED light or other device (448) exposed on the surface ofthe gripper may be controllably turned on or off with timing precisionby virtue of the realtime network and associated motor controller board(442)—while camera device triggering, as described above, is conducted.With the LED light in the field of view of the triggered camera, suchconfiguration may be utilized for calibration. In one embodiment, forexample, camera and LED lighting commands may be mixed (say to have theLED light up on the first 3^(rd) and 5^(th) milliseconds of a givencamera exposure, etc, so one can get a higher rate of visual data thanthat at which the camera is actually capturing, and see a flashingdot/LED as the gripper moves in the captured images, to understand wheresuch dot/LED is in space and use this information for calibration and/orregistration). In another embodiment, all cameras may be exposedsynchronously, due to the precision timing of the realtime network.

Another variation of a motor controller board resides in the headassembly (12) and is configured to be operatively coupled to one or moretexture projector devices (164), for millisecond control over relatedprojections, as well as a 6 degree of freedom inertial measurement unit,or “IMU”, which may be coupled to the lower head assembly (128) andconfigured to synchronously measure rotation speed and accelerationalong any of three axes. The IMU may be a MEMS-based unit, such as thoseavailable from MicroStrain, Inc. In one embodiment, the projector andmotor controller board may be utilized to provide texture projection forstereo camera configuration depth perception or other image analysis,and such projected texture may be projected at precisely the same timeas images are captured with the cameras, or intentionally out of syncwith certain captured images, so that such images would not be cloudedwith the texturing. Many useful related timing configurations may beimplemented with the realtime control of such devices. For example, ithas been shown that humans tend to feel “seasick” if they observeflashing lights at about 30 Hz. One of the things that can be done withthe subject system to avoid such issues, is to run the texture projectorat 60 Hz or 120 Hz, and do more texture pulses than camera exposures.The camera(s) may be run in sync, out of sync, or partially in sync withthe projector(s). In one syncing embodiment, three modes may beutilized: 1) gate the projector and image capture together; 2) gate themoppositely; or 3) alternate between gated together and gated oppositely.The first mode can be desirable, for example, in a scenario wherein onehas a camera configured to capture images at 30 Hz, and texture isprojected at 15 Hz; in such a case, at 15 Hz, one would capturealternating 3-dimensional image data and texture data without corruptionof either. In a scenario with two cameras, each at 30 Hz, captures ofeach may also be timed around texture projector pulses, and with thehigh level of timing precision available using the realtime computingconfiguration, there is enough timing room to move certain projectorpulses forward in time (i.e., out of their conventional normal rhythmiccycle position) to avoid the human blurriness or seasickness issues thatmay be associated with more regular, and less customized, timingpatterns. In other embodiments, other similar devices and/or sensors maybe precisely controlled utilizing the subject control system, includinglaser scanners, stereo cameras, mono cameras, and depth cameras thatmeasure depth at every pixel (such as those available under thetradenames “Kinesta”, “SwissRanger”, and “PDM”).

While in one embodiment the subject robotic system may be controlled andoperated mainly using forward kinematics at the joints, in anotherembodiment, vision-based systems and absolute sensing may be utilized tounderstand where various portions of the robot are in space—for example,where the upper arms, forearms, and/or grippers are. In one embodiment,imaging fiducials may be utilized to facilitate observation of variousportions. In another embodiment, shape recognition of the various partsof these assemblies, using a dense three-dimensional textured rigid bodymodel, for example, may be utilized to understand their positions inspace. Such techniques may be utilized either to enhance forwardkinematic based techniques, or to at least partially replace suchinfrastructure, providing the opportunity to have less wires and sensorson certain portions of the robot. Further, imaging and scanning systemsmay be utilized to better understand the environment external to therobot. For example, given the significant amount of runtime envisionedfor the subject robot within human environments, such robot isconfigured to have the opportunity to learn its environment whilenavigating through various locations. Simultaneous localization andmapping, or “SLAM”, techniques may be utilized to further advance suchrobot's mapping and understanding of its environment while navigatingand localizing various objects in real or near real time.

Referring again to FIG. 11A, various master input devices may beutilized to remote control or teleoperate aspects of the subject roboticsystem. For example, in one embodiment, a Bluetooth-enabled joystick(536) may be operatively coupled to one of the control system computers(462) through a Bluetooth adaptor (530) and USB interface (526) tofacilitate the passage of instructions from an operator of such joystickto the control system. Similarly, a user interface of a PDA device, suchas the touchscreen of an iPhone® or iPad®, may be utilized to provideinstructions to the robotic system via Bluetooth/USB. Alternatively,such WiFi enabled PDA devices may be connected to one or more controlsystem computers (462) using the service wireless access port (500).Many configurations may be utilized, depending upon the latencyrequirements. For example, in another relatively low latency embodiment,a master input device (510), such as the multi-degree-of-freedom devicesavailable under the tradename “Phantom®” by Sensable Technologies, Inc.,may be operatively coupled to a base computing station (508) positionedoff-board the robotic system, and such base station (508) may beconnected via VPN (508, 506) and a fast network connection which may,for example, be part of a local building infrastructure, to a centralcomputing system (462), preferably through an equally fast interface onthe network port infrastructure, such as an rj45 network port (496), ora high-speed network wireless access point (454) (a SIM-card typeconnection 498 to a mobile network presumably would be slower and havegreater latency, but would indeed function). In an embodiment wherein anoperator is close to a robot but does not want to, or is unable to,connect to high speed local building infrastructure or the like, one ofthe easy ways to use an external master input device is through theservice port infrastructure (500, 502), as described above in referenceto the iPhone® and iPad® devices as master input devices.

Referring again back to FIG. 11A, in one embodiment the controlsarchitecture is all on Ethercat, so it is all time synchronous. Acomponent may simply be operatively coupled with one of the motorcontrol boards which exist all over the subject robotic systeminfrastructure (for example, in the aforementioned embodiment, aprojector is operated by the same motor control board that controls acamera and a motor), and it becomes part of the control network. Asdescribed above, on the hardware side, Ethercat looks similar toEthernet; on the software side, Ethercat is a unique ethertype, withframes being dispatched having addressing information dictating whereone wants to read from and write to in a process image; this processimage is distributed out to the slave motor controller boards (each ofwhich preferably is configured to have its own address). Packets may bedesigned to go out on the network and distribute portions to variousmotor controller boards in a given configuration. For example, packetsmay be designed to go out, set every device's set points, read data fromeach, then capture such data on the way back and bring it in as a singlepacket. And, importantly, all of this may be handled in real time.Indeed, with a fast network and a 1 millisecond frame, data may startbeing received from such a dispatch before the dispatch has beencompleted, because the packet may be processed “on the fly” in less thanhalf a millisecond. The remainder of the frame may be used forcomputation, as well as other tasks, such as “mailbox read”, sending andreceiving other information, such as temperature information or otherdata related to the motor model.

Once packet information from the motor controller boards makes it's wayback to a central processing computer (462) by Ethercat (461), a seriesevents occur in the embodiment depicted in FIG. 11B. Inbound sensor data(468) is delivered to an Ethercat hardware driver (470) which isconfigured to convert electronic wire transmissions into data structuresthat can be used by the controllers. These data structures are thenpassed to a hardware interface layer (472) representative of manyhardware aspects. Everything beyond this layer on the inbound side isEthercat independent. The hardware interface (472) has the same level ofmodularity used in a coding simulator to represent motors, encoders,joint range of motion limit flags, series elastic joints, positionsources, torque sources, digital outputs, triggers, projectors, etc—asmany different pieces of hardware as need to be communicated with.Multiple Ethercat buses may be configured to feed in this information,or one Ethercat bus may be configured to handle portions of it, whileworking with a more conventional second bus to handle other portions—away of unifying to enable the control level to be the same. Indeed, withthe subject system architecture, multiple totally disparate controlsystems may be controlled synchronously together. In the descriptionabove of the motor controller boards, on-board storage was described.Preferably in this storage is everything needed to not damage thehardware, along with hardware configuration information. So, the systemknows based upon this information, for example, that it is dealing withthe left forearm motor, and has the motor electrical parameters, thetorque constant, the resistance, the motor model number, etc—all of theinformation needed to model the motor system, and all of it is stored onthe motor controller board. This is valuable, because a new componentdesigned using the same configuration, such as a new robotic arm, may beplugged onto the robotic system and the remainder of the systemunderstands how to run it and not damage the hardware.

Referring back to FIG. 11B, information comes in from the motorcontroller boards (468), is converted into a useful form by the Ethercathardware driver (470), is interpreted with the hardware interface layer(472), and is checked for motor model enforcement, among other things.For example, referring (479) to the motor model and an XML descriptionof the entire robotic system (kinematics, dynamics, gear reductions,etc—the entire kinematics of the system) which may be referred to as the“URDF” (480), checking is conducted for back EMF, commanded sweep, inputvoltage (input voltage multiplied by pulse width modulation duty cycleshould be approximately equal to the measured output voltage),commanded/measured current equivalence (they should be approximatelyequal unless there is a problem), and generally to confirm that themotor model remains intact (in one embodiment, a complex motor modelequation may be utilized for such checking). So the hardware interface(472) is configured to do a lot of checking. After all of this checkingand confirming, the information exposes, like a more sanitized versionof the motor that meets control theory expectations (for example, wherea torque is commanded and a position is returned). Beyond this analysis,there is a group of actuators represented in the hardware interface.Subsequent to the hardware interface layer (472) processing, theinformation is passed to what may be called “mechanism control”.

Referring to FIG. 11B, mechanism control also runs its own battery ofsanity checking on inbound information to confirm that the data looksreasonable and that aspects of the robot are not powered down. Then itis configured to run through a series of controllers (478), which arerepresentative of almost any kind of pluggable code that one wants runthrough the realtime control loop. So the system is configured to runthrough a realtime update loop on each of the controllers, and thecontrollers are able to refer (479) to the robot model and URDF (480),as described above in reference to the hardware layer, to check thingssuch as, “where is this joint”, “how long is this link”, etc. In oneembodiment, data comes into the controllers (478), an update function isrun, and the data runs out. All of this is happening once permillisecond, and once the controllers are loaded in, they aredynamically loaded (from anywhere—so they may be unloaded, recompiled,reloaded, initialized, and restarted at runtime) into the singlerealtime process. The controllers also are configured to be able to runand startup non-realtime threads, and make ROS or other types ofcommunication layers. They are also configured to publish, subscribe,and do all sorts of non-realtime tasks, in addition to updating theloop. They may be configured to be responsible for their owncommunication between the realtime and non-realtime processes, whichmeans that they can get access to whatever kind of realtimecommunication data they need to get. They also are configured to publishdata out and receive set points in a variety of different ways. In otherwords, a custom interface may be defined that is appropriate forwhatever task is at hand, and typed information may be published andoperated along with the rest of the tasks, even though all are in thesame process. For example, in a robotic odometry scenario, a controllermay be created that is called the “odometry controller”. It has accessto the robot model (480) to understand where the casters of the mobilebase are located, and it uses all of the data coming in (461) tounderstand how the casters (and joints and wheels thereof) are moving.With this information, it can be configured to then publish odometryinformation equivalent to “here is where the robot actually is in theworld”.

Thus, various controllers get instantiated and message to the rest ofthe system what kinds of type they want to load, and what kind ofconfiguration data is required (for example, from a parameter serverassociated with the ROS operating system). The configuration data isacquired, dynamically loaded, initialized, and run. In one configurationthere is a plurality of IPC service calls configured to allow a returnon a query regarding what controllers are currently running, what typesof controllers can be run, how to stop and start various controllers,etc. Preferably there are different controllers configured to implementdifferent control schemes. For example, in one embodiment, jointcontrollers are included, as are joint trajectory controllers, Cartesiancontrollers, force controllers, hybrid controllers, and customcontrollers for testing and analysis outside of normal operation (suchas a realtime testing controller configured to run one or moresinusoidal joint sweeps, followed by analysis, and production of aFourier transform).

Referring to FIGS. 11C and 11D, part of a built in safety architecturein the depicted embodiment is a series of torque, velocity, and positionrelationships pertinent to every joint in the system (in one embodiment,everything has a torque limit, everything has a velocity limit, andcertain things have position limits—while others, such as infiniterotation slip ring roll joints, intentionally do not). Referring againto FIGS. 11C and 11D, in short, an incoming commanded torque is comparedagainst allowable maximum and minimum torques that are part of apredetermined profile (484), and the commanded torque is truncated to bewithin the profile—but before this is completed, the profile (484) isalso analyzed for velocity constraints of the profile (484), andassociated with this is a similar predetermined position-velocityprofile (482) configured for truncating commands relative thereto.

In greater detail, referring to FIG. 11C, a profile (482) is shown whichallows a given joint to run at relatively high velocity (angular orrotational velocity, for example) until close to the ends of a givenrange of motion (position), wherein allowable velocities are truncated.The same thing is done for torque versus velocity in FIG. 11D. If ajoint of the robot is at or near maximum velocity, the torque which maybe applied will be scaled down as per the predetermined profile (484).The interworkings of such profiles (482, 484) are safety features whichprevent the robot from breaking itself or harming other objects aroundit. They are also advantageous because the system does not needadditional control gains or control gain interactions between regularcontrollers and safety limits—because in the depicted embodiment, thesystem is always doing a combination of maxing out and truncating. Theconfiguration has a transition between two controllers, both of whichare stable (because at the transition, it is known that they are forcecontinuous since it is a maximum, so a force continuous transition isconducted between two stable controllers, and you end up with positiondependent velocity limits and velocity dependent torque, or force,limits). Operating under such paradigm, a joint can be moved all of theway up to the edge of a limit gently and slowly, but is not allowed tosmash with a high impulse into the mechanical limits. Thus, the robot isable to have use the entire safe working envelope without damagingitself or other objects. With such active joint limits (i.e., withsoftware slowing the mechanical interfaces before the hit mechanicalstops), in one embodiment the software generally should not be pushingpast a “soft” joint limit. If a given joint does, the system may beconfigured, for example, to shut down mobile base assembly (4) movement,or any movement of any joints. In one embodiment, the robotic system maybe configured to temporarily push through software-based limits such asthose described above—say in a scenario wherein there is a need totemporarily go all the way to the mechanical limits. In such a mode, therobot may be configured to provide feedback to an operator or nearbypeople that it has used a mechanical stop, such as a beep sound or evena voice expression, such as “ouch”. In another mode, the system may beconfigured to keep the joint limits very well within the mechanicallimits—say 50% within the mechanical limits—to allow for plenty ofbackdriveability in any direction of the joint motion, such as in anextra safe mode wherein collisions with other objects or people are morelikely. In another mode, such as in a controlled industrial workenvironment, the system software may be configured to keep the jointlimits within 90% of the mechanical limits—to allow for greater range ofmotion, while still retaining software limits configured to stop thejoint motion short of mechanical limit enforcement. In anotherembodiment, the robotic system may be configured to fold its arms andessentially go “limp” (the aforementioned mechanical gravitycompensation would continue to gently levitate the arms) when a child orother object or being is suspected to be in the nearvicinity—significantly limiting the ways in which such child or otherobject could be injured or damaged through a collision with the roboticsystem.

In one embodiment, other safety features are built into the systemconfiguration to guarantee that motor actuation will shut down if thecurrent loop has a problem that may cause hardware damage, or ifcommunications with the control computing systems have been interrupted(a “watchdog” type configuration is utilized to send dispatches to allpieces of hardware and wait for responses back). In one embodiment, if a“watchdog” timeout happens at the motor controller board level, themotor controller boards continue to run (i.e., to report all pertinentinformation), but not to operate associated motors—a flag is triggeredwhen something is disabled, and the system may be configured to disablethe entire associated chain, depending upon what associated devices maybe safely actuated notwithstanding the given flag. Such a flag may beremoved in software (generally a relatively high level software requestis required to ensure that it is more than trivial to handle, andcertainly intentional). One way to handle this is to have an “enable”bit in the Ethercat hardware, such that when this bit is high, thesubsystem is enabled, and when low, it is disabled. If a fault isgenerated, it must be toggled low, then high again to ensure that nomatter how many packets are dropping, or whatever else is the problem,that there has been active acknowledgement from the central processingside that a re-enablement is indeed warranted. Once this has beenconfirmed, there are other checks going on, such as communication packetcheck sum errors. In one embodiment, the system is configured to runthrough a first packet check sum error and treat this first packet aslost, but if there are too many (i.e., past a certain predeterminedthreshold—using a tally that remains updated), a fault may bedetermined. Similarly, motor model warnings (for example, due to thermalor undervoltage issues) may be initially reported as warnings, but pasta certain threshold a fault may be determined, the motors shut down, andthe issue reported out to the rest of the system on a diagnosticsreporting interface.

Referring to FIG. 11E, and the above described Estop (24) functionality(either from the Estop button 24 positioned at the back of the bodyassembly 8, or an Estop button which may be positioned within a wirelessremote controller housing and configured to have precisely the samefunctionality of the body assembly Estop; the system may be configuredto execute an Estop if watchdog type functionality doesn't recognizethat such wireless Estop is within range and functioning properly) thesystem preferably is configured to operate using an unregulated batteryvoltage from the batteries housed within the mobile base assembly (4),the voltage varying from about 44 volts to about 80 volts. In theexample of FIG. 11E, if the robotic system is operating at approximately75 volts and an operator hits an Estop button (24), the system isconfigured to immediately drop the voltage to approximately 18 volts andcontinue at that level. This voltage level is configured to representjust enough power to run all of the motor controller boards in a standbymode wherein, based upon the boost circuitry configuration of the motorcontroller boards, they cannot power the FETs and operate associatedmotors unless they have more than 18 volts. This is advantageous,because two modes of operation are available without two separatesources of power (i.e., one for communications and one for motoractuation), which would require twice as many wires, more weight, andthe introduction of electrical ground loops (in one embodiment, thecommunication is Ethernet—so this is also decoupled and does notintroduce ground loops). Upon commanded reenablement, as shown in theplot (486) of FIG. 11E, voltage is increased back to the previousposition (75 volts in this example), and the motor controller boards maybe configured to wait for an affirmative signal from the central controlsystem (i.e., a conscious reset bit after the voltage is turned back tohigh) before full motor actuation enablement.

As described above in reference to the mobile base assembly (4) and bodyassembly (8), the subject robotic system has a mobile power systemcomprising a plurality of batteries (44), wiring interfaces, and a mainpower control board (58). In one embodiment, power to charge thebatteries may be brought in through a conventional cable and plug from astandard AC wall outlet to an International Electrotechnical Commission(“IEC”) specification 60320 type of plug interface featured at the sideof the mobile base (4) in the power switch panel (28) area. In oneembodiment, the robot may be configured to plug itself in using the IECinterface, a cable, and a conventional wall outlet. In anotherembodiment, the robot may be configured to analyze how much power can bedrawn without overdrawing and toggling a conventional circuit breaker.In another embodiment, the robot may be configured to draw at arelatively low predetermined level, such as 5 amps, unless otherwiseconfigured. With a cable in place, power comes in through the IECinterface and into AC/DC converters housed within the mobile baseassembly (4). As described above, the robot may be configured to operateusing unregulated battery voltage for efficiency reasons (conventionaldevices regulate voltage first, and then use this to run current controlwith a current regulator—two regulators in series is not as efficient).Charge management boards are operatively coupled between the AC/DCconverters and batteries in a configuration not unlike those of laptopcomputers. Such charge management boards preferably are operativelycoupled to the control system computer so that it may monitor chargestatus, time to charge, whether the system is indeed plugged into a walloutlet that is providing power, etc. In one embodiment, the AC/DCconverters also are operatively coupled to the control system computingresources so that they may also be monitored. When one of the Estopbuttons, either wired or wireless, is activated, the voltage is pulleddown and the motors stopped, as described above. With the abovedescribed backdriveable arm assemblies (14, 16, 18), the robot may beconfigured to produce power regeneratively, when work is being done onthe robot. For example, in the case of carrying heavy objects in afactory setting from a first position relatively high above the ground,to a second position relatively close the ground, gravitational forcesare doing work on the robot, and this work may be converted to energythrough regeneration, akin to the regenerative braking technologiesutilized in vehicles such as that marketed under the tradename “Prius®”by Toyota Motor Company of Japan. In one embodiment, the system isconfigured to be about 90% efficient when work is being done upon it.

Since the robot comprises a relatively sophisticated power supply, itmay be used as such by other devices, through the power switch panel(28) and associated plug interfaces. In one embodiment, laptops or otherdevices may be plugged into the robotic system during power outages orother scenarios wherein a mobile power supply is advantageous. Indeed,in one embodiment, the robotic system may be configured to navigate overand flip a breaker switch to reestablish local building power when suchbreaker has temporarily terminated local power, in one variation afterthe robot has conducted certain testing to confirm that the resetting ofsuch breaker seems warranted and safe. The power control anddistribution board (58) housed within the body assembly (8) isconfigured to be available over the non-realtime network to the controlcomputers and therefore may be accessed externally, for example througha laptop computer or remote basestation computer, through means such asthe service port infrastructure. In another embodiment, components suchas AC/DC converters and charge controllers may be featured off-board therobot to save space and mass, and have less heat generated on the robot.

As described above, the subject robotic system may be teleoperated usingone or more master input devices, such as a joystick, wireless PDAinterface, or mouse or other device associated, for example, with acomputing base station. While literal teleoperation can be very useful,it is also desirable in certain scenarios to work with higher-levelcommands. For example, as opposed to a simple teleoperation command suchas “move forward”, a command such as “go and get me the newspaper” or“clean the kitchen” may be desired. Teleoperation can play an importantpart in certain embodiments of higher-level instruction response orvarious levels of autonomous operation. For example, it may be useful tobring external teleoperation and decisionmaking resources into a givenrobotic system challenge to assist with efficient net goalaccomplishment. In one embodiment, external human teleoperationresources may be integrated into the operation of a given robotic systemto accomplish and instructed task such as “clean up the kitchen.” In oneembodiment, outside resources such as human teleoperators may beutilized to assist with robotic decisionmaking in a relationship to therobotic system similar to that established for the participants of“mechanical turk” arrangements, such as that operated by AmazonMechanical Turk (www.mturk.com). For a given cost, a question or issuemay be answered by an external, but connected, human resource. Overtime, a database may be populated with successful answers to givenqueries, and certain answers may be delivered from the database withoutoutside human intervention. For example, a human resource may be broughtin to assist as a teleoperation resource in a scenario wherein a roboticsystem has been given a high level assignment, such as “load thedishwasher”, and may have difficulty identifying which dish to pick upfirst, which items are dishes versus drinking glasses, etc. Ateleoperator in such a scenario may use the imaging resources and hisown brain to identify the dishes and glassware, along with theirorientations and positions, and then use this information to assist inoperation of the robot—say with a master input device such as a joystickto identify which place to grasp next as the robot is moving through theprocess of picking up dishes and glassware. Similar teleoperationassistance may be provided when an autonomous or semiautonomous roboticsystem is trying to figure out how to open a door that has a uniquepiece of doorknob hardware, or one that is fairly common to the averagehuman, but is difficult for the robot to interpret or operate. For afirst plurality of attempts through such a complex task, a mechanicalturk marketplace of human intervention may be utilized—and after adatabase has stored enough information regarding such tasks, itpreferably will be configured to utilize the resources of the databasefirst before pursuing other human intervention (for example, after arobot has brought in external human resource to open the doors of agiven building 1,000 times, it will “know”, by virtue of the database, alot about opening the doors of that building and will likely be able tohandle most scenarios on its own, without additional outside resources,such as that which maybe brought in from a mechanical turk type ofconfiguration). Further, it is desirable for a network of robots toassist each other—by sharing information. For example, if every robotmust master the art of opening doors on its own, a lot of resource willbe consumed doing so when looking at a group of 10,000 deployed robots.Conversely, in one embodiment, the robots may share information andresources, so that they can learn together how to open entry doors,refrigerator doors, and any other kind of door—and constantly refinetheir commonly accessible information regarding addressing suchchallenges in the human environment. In one embodiment, such informationavailable from a commonly accessible database, would be anonymized toaddress privacy issues. For example, in a scenario wherein the challengeis in opening a door by engaging a doorknob and mechanically swinging adoor, there is no need to also provide accessible information regardingthe addresses or home layouts of any particular homes. A result of suchinteraction of robotic system based database building and humanintegration, such as by mechanical turk types of configurations, is amarketplace of solutions and information pertinent to robotics operationin the human environment. Certain companies or organizations may be setup to build and advance these kinds of databases or address these kindsof human integration opportunities. Tasks may be weighted, in terms ofimportance to the robot or owner having a given query responded to andthe quality of the response. For example, a housecleaning robotoperating in a $40 M house full of valuable artwork may be configured byits owner to seek outside human intervention or designated very highquality database information any time it has confusion regarding what ithas encountered—whether it's a cat, a valuable statue, or a telephonecord stretched across a carpeted walkway. On the contrary, a junkyardorganizing robot may be configured to sort metals and plastics, and togenerally only seek outside help if something appears to be dangerous tothe robot itself. Trustworthiness and quality ratings, akin to thoseused on eBay or other collaborative business systems, may be utilized invarious embodiments to assist an owner or operator in determining whatkind of resource is available and how to increase the probabilities ofsuccess. For example, a teenager in Bangalore, India with a verysophisticated, multiple degree of freedom, master input device andproven high levels of mechanical acuity maybe rated differently forcertain tasks relative to a 75 year old retiree in Canada with an olddusty Atari joystick.

Outside resources also may be utilized controls paradigms pertinent tosafety and diagnostics. For example, in one embodiment, a commonlyaccessible database regarding combinations of messages, such as errormessages from motor controller boards, may alert referring systems thatsomething disadvantageous may be coming and can be avoided. For example,if a temperature appears to be going up in a location on a given roboticsystem wherein the voltage is also changing significantly, this may becorrelated, given past experience with other robotic systems, or perhapsthe system at issue, with a fire that is about to break out, which canbe avoided if the issue is identified early enough. Rules for combiningincoming information, such as temperature and voltage, may be developedat the robotic system diagnostics level to provide predictable andchangeable responses, and libraries of rules may be made available todisparately located robotic systems, in a manner akin to that used byoperations such as www.bigfix.com, wherein users of disparate computersystems are assisted with information technology challenges that havebeen encountered in the past and successfully addressed. For example, inone embodiment, when a robotic system is unable to operate a “model 15gripper” when it is also operating a “model 3” texture projector, therobot may query a database to leverage the experience of other robotsthat had the same problem and successfully addressed it with “SolutionX”—and may apply Solution X to its own scenario to try to address thegripper/projector issue. In another embodiment, the outside resource mayreturn a message indicating that some maintenance issue is required thatshould be addressed, such as a worn slip ring connection.

As described above, a robotic system may comprise various image capturedevices and/or projectors. The projectors may be configured to broadcasttextures for image capture and spatial analysis, or may be utilized asuser interface devices, for example, by projecting an image in two orthree dimensions for viewing by a person. For example, in oneembodiment, a movie clip or image may be projected against a wall. Inanother embodiment, the robot may comprise a two or three dimensionaldisplay or monitor which may be configured to broadcast images, signals,or the like to persons or other robots around the subject robot. Forexample, in one embodiment, a robotic system may be configured tobroadcast a laser dot onto something that it is about to pick up with agripper—to let others around it know that it is in the process of goingtoward that object to pick it up. In another embodiment, a laser dot orarrow may be broadcast onto the floor in front of a moving robot, toindicate to others where that robot is headed. In another embodiment, arestaurant waiter robot configuration may be configured to broadcast amenu image onto a flat table in front of one or more customers—and toallow the customers to point to items they want to order or have furtherinformation regarding (such as a list of ingredients, photos, or otherinformation which may then be displayed, verbalized, or otherwiseprovided by the robot to such customers). In another embodiment, akeypad image may be broadcast upon a table to allow a user to “type”with his fingers on the table to provide an input to the robotic system.In another embodiment, a display, such as a two or three dimensionalmonitor, comprising part of a robotic system may be utilized to indicateto others the charge status of a robot, whether the robot ismechanically stuck on something, whether the robot is attempting toautonomously solve a problem (i.e., whether it is paused and “thinking”about something), etc. In another embodiment, a robot may, in responseto an instruction to “go and get me a beer”, mobilize to therefrigerator, open the door, and upon identifying multiple types of beerin such refrigerator (i.e., by optical character recognition, shaperecognition, etc.), capture a photo of such scenario. Such photo may bethen be broadcast to the requesting person—say on a table, or wall, withthe person given a choice as to “which beer”. In a related embodiment,the robot may be configured to send the photo to the user's PDA ortelevision, so that the robot can stand in position at the refrigeratorand readily grab the correct beer after the user has selected it fromthe image using his PDA, laptop, television remote control (for example,the robot may be configured to interrupt regular television or videosignals and broadcast the captured image of the refrigerator, which maybe communicated, for example, to a WiFi-capable television from therobotic system), or other device as a selector or master input device(selections may be made, for example, by positioning with arrow keys ora touchpad interface, or by voice commands such as “up a littlemore—little more—yes, that one” or “the one with the yellow label”). Inanother related embodiment, the user may be given the opportunity tochange his mind or modify his command (i.e., “get me some crackersalso—those ones in the orange box on the top shelf . . . yes, thatbox”). In another embodiment, projection (such as by the robot, or by aconnected camera/projector, such as by an image capture device on a PDAor cellphone which may be connected) and a user's hands or one or moreindividual fingers of the user may be utilized as a master input device(using, for example, automated image processing to identify the fingeror fingernail positions, for example, or utilization of a glove orreadily identifiable markers coupled to the pertinent fingers—to enablethe robotic system to understand where the pertinent fingers are inspace relative to each other) to assist with robot control. For example,in a scenario wherein fine motor control is helpful, in one embodiment,the user may be able to use his thumb and forefinger, along with theimage capture device in his cellphone, to delicately simulate a pinchingpickup of a single olive from a tray of olives near his robot in anotherroom far out of sight—and the robot may be configured to follow thisfine motor control master input device command to pick up such olive forthe user. In another embodiment, a two-finger pinching type of activitymay be captured using a touchscreen interface, such as on an AppleiPhone®, to send a similar command to the robot; in a relatedembodiment, an image of the olive tray may be shown in the background ofthe iPhone as the user is allowed to simulate the selection and pickupof his targeted olive with the touchscreen interface overlaid over theimage. In another embodiment, the robot may comprise button pads, or akeyboard, for direct user interface engagement. Any of the buttons,touchscreens, or other user interface features of a PDA, laptop, orother communication device may be utilized to communicate with therobotic system. For example, a person trying to stay quiet, or perhaps adeaf person, may, in one embodiment, prefer to instruct his robot to gethim a beer by sending a text message or email to his robot with the sameinstruction. In another embodiment, the arms of the robot may beutilized as master input devices, and one robot or arm may be utilizedas a kinematically similar master input device for another robot or arm.The robotic system may comprise directional microphones, or one or moremicrophones and a directionality capability, to localize sounds. Thus inone embodiment, when a cat meows near a user, the user may query to hisrobotic system, “which cat was that” and the robot, with directionalsound sensing, may localize the meow sound, capture an image of thesubject cat, and communicate that to the user by any of theaforementioned user interface techniques (for example, an image of thecat's face may be sent by Bluetooth to the user's PDA screen, sent byWiFi to the user's television or laptop, broadcast upon a nearby wall,broadcast upon a display coupled to the robot, etc). Similarly, if theuser says, “where is he?”, the robot may be configured to broadcast alaser pointer spot near the position of the cat, point with the roboticarm in a manner akin to a human pointing with his finger, etc. Inanother embodiment wherein the cat has a Bluetooth or other networkenabled collar, the robot may be able to localize the cat's positionusing signal triangulation, and directly identify the cat using thecollar-based information (i.e., the robot may be configured to audiblysay, “it's Fluffy” and/or broadcast a stock photo or captured image tothe user). In another embodiment, multispectral sensing hardware andsoftware may comprise portions of the robotic system, and may beconfigured to resolve objects in somewhat unconventional ways. Forexample, in one embodiment, an X-ray tube may be coupled to the robotichead or arm assembly and configured to assist a carpenter withidentification of non-visible resources positioned within a wall—such aspipes, fuel or electrical lines, pressure vessels, beams or otherframing members, etc. In one embodiment, the positions of such resourcesmay be communicated to an nearby user in various ways as describedabove, such as by utilizing a projector to project an image of thein-wall resources upon the flat surface of the wall, or by using a lasersource coupled to a scanner, such as a galvanometric scanner, to producea high-frequency-recirculating trace of the resources that ends uplooking like a laser projection of an image against the wall to theuser.

With the capabilities and configurations described herein, the subjectrobotic system may be used in many other teleoperation, autonomous, orsemi-autonomous configurations and embodiments. For example, in oneembodiment, the robotic system may be configured to operate as a nannyfor children, with the capability to interact safely with kids, answerquestions, and provide enhanced monitoring capabilities for one or moreparents. In another embodiment, the subject robotic system may beconfigured as a construction contractor's assistant. In anotherembodiment, a subject robotic system may be configured as a real estateagent or property manager's assistant. In another embodiment, thesubject robotic system may be configured as a cook or cook's assistant.In another embodiment, the subject robotic system may be configured as anurse or surgeon's assistant. In another embodiment, the subject roboticsystem may be configured to operate as a searchable physical filingsystem assistant. In another embodiment, the subject robotic system maybe configured to operate as an elder care or handicapped person'sassistant. In another embodiment, the subject robotic system may beconfigured to operate as a manufacturing robot.

Various exemplary embodiments of the invention are described herein.Reference is made to these examples in a non-limiting sense. They areprovided to illustrate more broadly applicable aspects of the invention.Various changes may be made to the invention described and equivalentsmay be substituted without departing from the true spirit and scope ofthe invention. In addition, many modifications may be made to adapt aparticular situation, material, composition of matter, process, processact(s) or step(s) to the objective(s), spirit or scope of the presentinvention. Further, as will be appreciated by those with skill in theart that each of the individual variations described and illustratedherein has discrete components and features which may be readilyseparated from or combined with the features of any of the other severalembodiments without departing from the scope or spirit of the presentinventions. All such modifications are intended to be within the scopeof claims associated with this disclosure.

Any of the devices described for carrying out the subject diagnostic orinterventional procedures may be provided in packaged combination foruse in executing such interventions. These supply “kits” may furtherinclude instructions for use and be packaged in sterile trays orcontainers as commonly employed for such purposes.

The invention includes methods that may be performed using the subjectdevices. The methods may comprise the act of providing such a suitabledevice. Such provision may be performed by the end user. In other words,the “providing” act merely requires the end user obtain, access,approach, position, set-up, activate, power-up or otherwise act toprovide the requisite device in the subject method. Methods recitedherein may be carried out in any order of the recited events which islogically possible, as well as in the recited order of events.

Exemplary aspects of the invention, together with details regardingmaterial selection and manufacture have been set forth above. As forother details of the present invention, these may be appreciated inconnection with the above-referenced patents and publications as well asgenerally known or appreciated by those with skill in the art. The samemay hold true with respect to method-based aspects of the invention interms of additional acts as commonly or logically employed.

In addition, though the invention has been described in reference toseveral examples optionally incorporating various features, theinvention is not to be limited to that which is described or indicatedas contemplated with respect to each variation of the invention. Variouschanges may be made to the invention described and equivalents (whetherrecited herein or not included for the sake of some brevity) may besubstituted without departing from the true spirit and scope of theinvention. In addition, where a range of values is provided, it isunderstood that every intervening value, between the upper and lowerlimit of that range and any other stated or intervening value in that zostated range, is encompassed within the invention.

Also, it is contemplated that any optional feature of the inventivevariations described may be set forth and claimed independently, or incombination with any one or more of the features described herein.Reference to a singular item, includes the possibility that there areplural of the same items present. More specifically, as used herein andin claims associated hereto, the singular forms “a,” “an,” “said,” and“the” include plural referents unless the specifically stated otherwise.In other words, use of the articles allow for “at least one” of thesubject item in the description above as well as claims associated withthis disclosure. It is further noted that such claims may be drafted toexclude any optional element. As such, this statement is intended toserve as antecedent basis for use of such exclusive terminology as“solely,” “only” and the like in connection with the recitation of claimelements, or use of a “negative” limitation.

Without the use of such exclusive terminology, the term “comprising” inclaims associated with this disclosure shall allow for the inclusion ofany additional element—irrespective of whether a given number ofelements are enumerated in such claims, or the addition of a featurecould be regarded as transforming the nature of an element set forth insuch claims. Except as specifically defined herein, all technical andscientific terms used herein are to be given as broad a commonlyunderstood meaning as possible while maintaining claim validity.

The breadth of the present invention is not to be limited to theexamples provided and/or the subject specification, but rather only bythe scope of claim language associated with this disclosure.

1. A humanoid robot, comprising: a. a mobile base; b. a verticallyextensible torso assembly movably coupled to the base; c. one or morerobot arms movably coupled to the torso assembly; wherein each of theone or more robot arms may be suspended in a substantially neutralconfiguration versus affects of the acceleration of gravity by aspring-balanced gravity compensation mechanism that is notelectronically actuated.